Backpropagation for long sequences: beyond memory constraints with constant overheads

Naive backpropagation through time has a memory footprint that grows linearly in the sequence length, due to the need to store each state of the forward propagation. This is a problem for large networks. Strategies have been developed to trade memory for added computations, which results in a sublinear growth of memory footprint or computation overhead. In this work, we present a library that uses asynchronous storing and prefetching to move data to and from slow and cheap stor- age. The library only stores and prefetches states as frequently as possible without delaying the computation, and uses the optimal Revolve backpropagation strategy for the computations in between. The memory footprint of the backpropagation can thus be reduced to any size (e.g. to fit into DRAM), while the computational overhead is constant in the sequence length, and only depends on the ratio between compute and transfer times on a given hardware. We show in experiments that by exploiting asyncronous data transfer, our strategy is always at least as fast, and usually faster than the previously studied "optimal" strategies

F-15B Quiet Spike Aeroservoelastic Ground and Flight Test

This viewgraph presentation reviews aeroservoelastic analyses of the F-15B Quiet Spike aircraft that includes ground and flight tests

Structure Detection of Nonlinear Aeroelastic Systems with Application to Aeroelastic Flight Test Data

This viewgraph presentation reviews the applicability of NARMAX structure detection to aeroelastic systems. In conclusion, the simulation results demonstrate bootstrap approach for structure computation of aircraft structural stiffness provided a high rate of true model selection: 1. T-test and stepwise regression methods had difficulty providing accurate results 2. Work contributes to understanding of the use of structure detection for modelling and identification of aerospace systems. 3. Limitation of model complexity that can be studied with these structure computation techniques 4. Result of the large number of candidate terms, for a given model order, and the data length required to guarantee convergence 5. Another approach to structure computation problem uses a least absolute shrinkage and selection operator (LASSO

X-Ray Detection of Transient Magnetic Moments Induced by a Spin Current in Cu

We have used a MHz lock-in x-ray spectro-microscopy technique to directly detect changes of magnetic moments in Cu due to spin injection from an adjacent Co layer. The elemental and chemical specificity of x-rays allows us to distinguish two spin current induced effects. We detect the creation of transient magnetic moments of $3\times 10^{-5}$ $\mu_\mathrm{B}$ on Cu atoms within the bulk of the 28 nm thick Cu film due to spin-accumulation. The moment value is compared to predictions by Mott's two current model. We also observe that the hybridization induced existing magnetic moments on Cu interface atoms are transiently increased by about 10% or $4\times 10^{-3}$ $\mu_\mathrm{B}$. This reveals the dominance of spin-torque alignment over Joule heat induced disorder of the interfacial Cu moments during current flow

A LEAST ABSOLUTE SHRINKAGE AND SELECTION OPERATOR (LASSO) FOR NONLINEAR SYSTEM IDENTIFICATION

Identification of parametric nonlinear models involves estimating unknown parameters and detecting its underlying structure. Structure computation is concerned with selecting a subset of parameters to give a parsimonious description of the system which may afford greater insight into the functionality of the system or a simpler controller design. In this study, a least absolute shrinkage and selection operator (LASSO) technique is investigated for computing efficient model descriptions of nonlinear systems. The LASSO minimises the residual sum of squares by the addition of a 1 penalty term on the parameter vector of the traditional 2 minimisation problem. Its use for structure detection is a natural extension of this constrained minimisation approach to pseudolinear regression problems which produces some model parameters that are exactly zero and, therefore, yields a parsimonious system description. The performance of this LASSO structure detection method was evaluated by using it to estimate the structure of a nonlinear polynomial model. Applicability of the method to more complex systems such as those encountered in aerospace applications was shown by identifying a parsimonious system description of the F/A-18 Active Aeroelastic Wing using flight test data

Direct observation and imaging of a spin-wave soliton with p−p-like symmetry

The prediction and realization of magnetic excitations driven by electrical currents via the spin transfer torque effect, enables novel magnetic nano-devices where spin-waves can be used to process and store information. The functional control of such devices relies on understanding the properties of non-linear spin-wave excitations. It has been demonstrated that spin waves can show both an itinerant character, but also appear as localized solitons. So far, it was assumed that localized solitons have essentially cylindrical, $s-$like symmetry. Using a newly developed high-sensitivity time-resolved magnetic x-ray microscopy, we instead observe the emergence of a novel localized soliton excitation with a nodal line, i.e. with $p-$like symmetry. Micromagnetic simulations identify the physical mechanism that controls the transition from $s-$ to $p-$like solitons. Our results suggest a potential new pathway to design artificial atoms with tunable dynamical states using nanoscale magnetic devices

Interaction between personality traits and cerebrospinal fluid biomarkers of Alzheimer's disease pathology modulates cognitive performance.

During adulthood, personality characteristics may contribute to the individual capacity to compensate the impact of developing cerebral Alzheimer's disease (AD) pathology on cognitive impairment in later life. In this study we aimed to investigate whether and how premorbid personality traits interact with cerebrospinal fluid (CSF) markers of AD pathology to predict cognitive performance in subjects with mild cognitive impairment or mild AD dementia and in participants with normal cognition. One hundred and ten subjects, of whom 66 were patients with mild cognitive impairment or mild AD dementia and 44 were healthy controls, had a comprehensive medical and neuropsychological examination as well as lumbar puncture to measure CSF biomarkers of AD pathology (amyloid beta1-42, phosphorylated tau and total-tau). Participants' proxies completed the Revised NEO Personality Inventory, Form R to retrospectively assess subjects' premorbid personality. In hierarchical multivariate regression analyses, including age, gender, education, APOEε4 status and cognitive level, premorbid neuroticism, conscientiousness and agreeableness modulated the effect of CSF biomarkers on cognitive performance. Low premorbid openness independently predicted lower levels of cognitive functioning after controlling for biomarker concentrations. Our findings suggest that specific premorbid personality traits are associated with cerebral AD pathology and modulate its impact on cognitive performance. Considering personality characteristics may help to appraise a person's cognitive reserve and the risk of cognitive decline in later life